Regardless of whether neural nets are simple or complex, they generally comprise a number of resources which can be grouped around three functions:
storage: this function concerns on the one hand the storage of synaptic coefficients characterizing synapses interconnecting two neurons and on the other hand the storage of functional states of the neurons, PA1 interconnection: this function concerns the allocation of one neuron to another neuron for the transmission and/or exchange of different control signals and data signals, PA1 processing: this function concerns the calculation of various data, notably the calculation of neural potentials of the neurons and the calculation of the updating of synaptic coefficients during the learning steps. PA1 the respective synapses are physically organized in at least one concatenation of respective uniform cells forming a data path between an entry and an exit, wherein a data input of each next one of the cells is connected to a data output of a preceding one of the cells for data transfer; PA1 each cell comprises between the data input and the data output: PA1 the neural processor comprises a bus means for communication of an instruction; PA1 the cells being connected in parallel to the bus means; PA1 the neural net comprises: PA1 the neural net comprises: PA1 the synapses are physically organized in said first concatenation, operative to produce the aggregate sum and at least one further concatenation, operative to produce a further aggregate sum and functionally arranged in parallel to the first concatenation; and PA1 the neural processor comprises an adding means between the concatenation and the further concatenation on the one hand and the exit on the other hand to linearly combine the aggregate sum and the further aggregate sum prior to applying the non-linear function.
In conformity with the most common prior art, a neural processor comprises a storage unit which, generally speaking, stores all synaptic coefficients, each synaptic coefficient being accessed by an address in the storage unit. The interconnection function is thus implicitly obtained by wiring according to rank (the addresses) of the synaptic coefficients in the storage unit: all synaptic coefficients of the same neuron are stored at known addresses (which are generally logically adjacent) so that it suffices to increment/decrement an address so as to render said synaptic coefficients available.
The resources allocated to the synapses in this case represent only the storage section, and the other functions must be executed in the processing unit. In order to obtain an advantage over pure software implementations, it is necessary to utilize some degree of parallelism; to this end, the processing unit actually consists of n processing sub-units, each of which performs l/n of the operations of the network. In order to limit the number of physical interconnections, a sub-unit is rigidly connected to l/n of the synaptic memory.
For the same reasons as stated to limit the number of interconnections, a sub-unit may not be engaged in processing for arbitrary neurons, independently of the synapses involved. The subdivision of the network into sub-units thus introduces relations between the synapses contained in a memory section and the neurons concerned by these synapses. For example, the number of synapses arriving at a given neuron may not exceed the length of a memory block assigned to a sub-unit.
This results in a poor efficiency of the use of the hardware. In order to simplify control, the number of calculation cycles of the sub-units is raised to the maximum number necessary for the most complex configuration permitted by the device. Moreover, given memory words will not be used because they relate to blocks corresponding to neurons having few synapses.
Attempts have been made to optimize the processing durations and to improve the proportioning of the hardware for various situations encountered.
In this respect reference is made to the document: "A wafer scale integration neural network utilizing completely digital circuits" by M. Yasunaga et at., IEEE INNS International Joint Conf. on Neural Networks II-213 (1989).
Referring only to the part considered relevant to the present invention, the cited paper describes a reduction of the number of interconnections arriving at the same neuron. To this end, the architecture is subdivided into neurons, a neuron then being a block receiving a single synapse at a given instant. From a shared bus this block successively receives, in time distribution, data relating to all synapses arriving at this neuron.
Thus, there are as many processing units and storage units as there are neurons. For the same neuron, a small memory stores all synaptic coefficients relating to this neuron. They are sequentially addressed as in the case of a non-distributed architecture. This mode of operation enables a reduction of the number of interconnections arriving at a neuron.
However, such an architecture remains rigid; specifically, it is not possible to modify, one unit after the other, the number of synapses connected to a neuron. Thus, this does not offer flexibility for attributing a new synapse distribution configuration to the neural processor.
The subdivision of the memory into modules which are accessible in order to execute the operations on the synapses thus imposes restrictions as regards the topology of the network. As a result, a network having an arbitrary topology, defined by a group of neurons and their interconnections, cannot always be accommodated on the hardware device, even if the total number of neurons and synapses is lower than the number that could be theoretically handled by the system.